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^An  important  problem  in  psychological  test  theory  is  the  development  of  a 
sound  method  for  determining  whether  a  test  which  purports  to  measure  the 
level  of  a  certain  ability  is,  in  reality,  significantly  contaminated  by 
one  or  more  other  abilities  displayed  by  persons  taking  the  test.  Because 
of  the  large  number  of  private  and  governmental  organizations  routinely 
using  tests  to  screen  people  for  the  levels  of  various  abilities,  this  prob¬ 
lem  of  assessing  the  dimensionality  of  a  test  is  of  great  importance.  The 
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solution  will  be  useful  in  settings  other  than  psychological  testing,  since 
the  problem  is  one  of  general  interest  and  should,  hence,  be  an  important 
addition  to  statistical  methodology  literature.  Described  in  this  paper 
is  an  approach  to  the  problem  of  finding  a  theoretically  sound  and  useful 
procedure  for  making  inferences  about  the  dimensionality  of''*$»f?he  ability 
parameter!,  or  more  precisely,  the  dimensionality  of  the  distribution  of  j£  . 
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A  Statistical  Procedure  for 
Assessing  Test  Dimensionality 


William  Stout 
University  of  Illinois 


An  important  statistical  problem  In  psychological  test  theory  ie  the  devel¬ 
opment  of  a  sound  method  for  determining  whether  a  test  which  purports  to  mea¬ 
sure  the  level  of  a  certain  ability  Is,  in  reality,  significantly  contaminated 
by  the  varying  levels  of  one  or  more  other  abilities  displayed  by  persons  taking 
the  te6t.  For  example,  is  a  test  of  mathematical  ability  contaminated  by  vary¬ 
ing  levels  of  verbal  ability  displayed  by  persons  taking  the  test  or  is  a  test 
of  reading  ability  contaminated  by  varying  levels  of  familiarity  with  middle- 
class  American  culture  displayed  by  persons  taking  the  test?  Because  of  the 
large  number  of  private  and  governmental  organizations  routinely  using  tests  to 
screen  people  for  the  levels  of  various  abilities,  this  problem  of  assessing  the 
dimensionality  of  a  test  is  of  great  importance. 

The  solution  will  be  useful  in  settings  other  than  psychological  testing, 
since  the  problem  is  one  of  general  interest  and  should,  hence,  be  an  important 
addition  to  the  statistical  methodology  literature.  Thus,  it  seems  appropriate 
now  to  give  a  careful  abstract  statement  of  the  problem,  independent  of  its  psy¬ 
chometric  context. 

Consider  sampling  units  from  a  population  and  applying  several  treatments 
to  each  sampled  unit.  Suppose  that  the  outcome  of  each  unit-treatment  combina¬ 
tion  is  either  success  or  failure.  Suppose  that  associated  with  each  unit  is  a 
parameter,  0  (the  ability  parameter),  which  determines  the  likelihood  of  each 
treatment  being  successful  for  that  unit.  Assume  that  the  dimensionality  of  6 
is  unknown  (the  precise  mathematical  definition  of  the  dimensionality  of  0  will 
be  given  below).  Thus,  for  each  unit,  dichotomous  random  variables  {U^}  are 

observed,  where  1  Is  the  treatment  Index.  Let  "treatment  characteristic 
curves"  {P^(*))  be  defined  by 

P1(§)  «  pi^  -  i|e  -  e)  -  i  -  pfUj  «  o|e  -  e)  ,  [U 

the  probability  of  treatment  JL  being  successful,  given  that  the  sampled  unit  has 
ability  8.  It  is  assumed  that  the  process  of  random  sampling  units  Induces  a 
probability  distribution  on  the  population  of  units  with  associated  random  vari¬ 
able  G. 

Purpose 

Described  in  this  paper  la  an  approach  to  the  problem  of  finding  a  theoret- 


ically  sound  and  useful  procedure  for  Baking  Inferences  sbout  the  dimensionality 
of  6,  that  lst  more  precisely,  the  diaenslonallty  of  the  distribution  of  Q.  In 
order  that  this  problea  be  veil  formulated  mathematically,  the  diaenslonallty  of 
$  needs  to  be  defined  precisely.  The  definition  (Levine,  1981)  that  is  used 
depends  on  the  asymptotic  behavior  of  "formula  sequences.”  To  define  a  linear 
formula  sequence,  a  linear  formula  score  must  first  be  defined. 

Definition  of  a  Linear  Formula  Score 

Given  the  outcomes  (Up  U2,  ...,  Un)  of  i»  treatments  resulting  from  a  sam¬ 
pled  unit,  a  linear  formula  score  16  a  score  of  the  form 

o  -  I  aJn)U  12] 

“  i-1  X  1 

provided  that 

..(n)  >  0,  ”  ..(o)  -  1  .  13) 

1  i«l 

Then,  a  formula  sequence  Is  a  sequence  of  linear  formula  scores  (a^»a2  9  •••» 

a  ,  .so)  such  that,  referring  to  Equation  2, 
n 

(n)  (n+1)  _  (n+1)  (n)  [lt) 

ai  ai*  ai  V 

for  all  i'  <_  n,  i  n,  and  n  ^  1.  The  content  of  Equation  4  is  that  the  contri¬ 
bution  of  a  treatment,  say,  relative  to  another  treatment,  6ay,  ,  is  the 
same  for  all  linear  formula  scores  aQ  for  vhich  n  >  i,  n  >_  i'.  The  prototype  of 

•  linear  formula  score  and  a  formula  sequence  is  the  proportion-correct 


I  U,  /n 
i-1  - 


{U.,(U.  +U,)/2,  ....  I  U./n,  ...) 
1  1  1  1-1  1 


respectively.  Levine's  (1981)  definition  can  now  be  stated  (below,  Var  [XjY] 
denotes  the  variance  of  X,  given  T): 

A  sequence  of  dichotomous  random  variables  {Up  Uj,  . .., 

Un,  ...}  1*  d.  dimensional  if  there  exist  <i  formula  se¬ 
quences  {h^n^J ,  {hin^l,  {h^n^}  such  that  for  every 

formula  sequence  {h^n  ), 
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V.r  [h(n)|hjn),  ....  h<n)]  -►  0  1 7J 

as  n  +  •;  and,  moreover,  no  smaller  £  works* 

Note  that  it  is  the  set  of  observables  {Uj,  02,  Un,  *..)  that  Is  d 

dimensional*  The  ability  6  is  not  observable  and  is  known  only  by  inference* 
Nonetheless,  let  it  be  said  that  §  is  d  dimensional,  meaning  that  a  ^-dimen¬ 
sional  random  vector  6  and  treatment  characteristic  curves  (P^*))  (the  condi¬ 
tional  distributions  of  the  D^fs  given  0)  can  be  constructed  to  specify  the 
joint  probability  lav  of  the  ^-dimensional  U^fs. 

Assessment  of  Test  Dimensionality 

A6  stated  above,  the  dimensionality  problem  is  of  particular  importance  in 
the  field  of  psychological  testing.  In  this  case,  the  units  are  persons. and  the 
treatments  are  test  items.  The  function  P^(*)  is  called  the  item  characteristic 

curve  for  the  ith  item.  The  administration  of  a  psychological  test  is  modeled 
S6  a  two-stage  experiment,  the  first  stage  yielding  J  randomly  sampled  persons 
and  the  second  stage  consisting  of  the  administration  of  I  fixed  test  items  (the 
test)  to  each  sampled  person.  In  this  manner,  dichotomous  random  variables 
{Ujj);  i  ■  1,  2,  ••*,  I;  j  *  1,  2,  ...,  J  are  generated.  The  basic  statistical 

assumptions  made  are  as  follows: 

1*  Experimental  Independence  of  persons*  The  appropriate  assumptions 
are  made  concerning  the  joint  distribution  of  the  that  corre¬ 

spond  to  the  psychometric  assumption  that  persons  are  randomly  sam¬ 
pled  from  a  very  large  population  and  that  sampled  persons  respond 
to  items  Independently  of  one  another* 

2*  Local  independence  of  items*  The  appropriate  probabilistic  assump¬ 
tions  are  made  concerning  the  joint  distribution  of  the  tUy)  and  9 

that  correspond  to  the  psychometric  assumption  that  for  each  person, 
his  or  her  responses  to  different  items  are  independent* 

Consider  again  the  example  of  the  introductory  paragraph,  that  of  a  "mathe¬ 
matics*  test*  It  might  be  that  while  6  is  assumed  to  be  a  one-dimensional  ran¬ 
dom  variable  measuring  mathematical  ability,  in  reality  0  is  two  dimensional 
with  the  first  dimension  being  mathematical  ability  and  the  second  dimension 
being  verbal  ability*  In  the  case  of  psychological  testing,  the  most  Important 
statistical  problem  concerning  dimensionality  is  to  test  H  :  d  «  1  vs*  A  :  d  > 

1*  Recently,  this  author  has  constructed  a  statistic  to  test  this  hypothesis 
and  to  be  further  used  as  an  index  that  estimates  the  amount  of  regularity  in 
the  data  attributable  to  the  multidimensionality  of  6* 

Illustration 

It  is  rather  easy  to  imagine  applications  in  other  fields*  As  an  illustra¬ 
tion,  suppose  that  medical  subjects  (the  units)  undergo  allergy  sensitivity 
tests  to  various  environmental  substances  (each  such  test  Is  a  treatment).  Sup¬ 
pose  that  the  result  of  each  test  is  scored  1  or  0,  depending  on  tAether  an  al- 
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lerglc  reaction  is  observed  or  not.  Let  different  values  of  the  parameter  6  be 
assigned  to  subjects  according  to  each  subject's  sensitivity.  Then,  Inferences 
about  the  dimensionality  of  0  become  meaningful  in  attempting  to  develop  a  clas¬ 
sification  scheme  for  allergies. 

Description  of  the  Statistic 

A  description  of  the  constructed  statistic  can  now  be  given.  In  doing  so, 
the  psychological  testing  language  of  items,  persons,  and  ao  forth,  will  be 
used. 


1.  The  test  being  administered  is  split  into  two  subtests  of  lengths  M 
and  n,  respectively.  Here,  n  should  be  considered  as  large  and  M  as 
possibly  not  large.  Let  fR  denote  the  proportion  correct  on  the 

second  subtest  of  items  M  +  1,  H  +  2,  ...,  M  +  n. 

2.  (0,1)  is  partitioned  into  intervals 

UAW  -  [0,1)  |8] 

k  n 

such  that 


max^{ width  (A^k^)}  ■*  0  as  n  -*■  •  . 
For  example,  let 
.  (k) 


k-1  k 

>V  [n*L 


1.2, 


tn*] 


[9] 


110) 


where  [x]  denotes  the  integer  m  such  that  m  x. 

3.  Persons  are  now  grouped  into  categories  according  to  the  following 
rule:  Assign  a  person  to  category  (k,n)  if  for  that  person 


f  €  A(k) 
n  n 


1.2, 


HD 


(Here,  1^  denotes  the  number  of  categories.)  Thus,  persons  are  as¬ 
signed  to  the  same  category  if  they  all  get  about  the  same  propor¬ 
tion  correct.  This  categorization  of  persons  is  the  only  use  made 

fk) 

of  the  second  subtest.  Let  Jv  7  denote  the  number  of  persons  in 
category  (k,n). 

4.  To  construct  the  test  statistic,  take  the  ratio  of  two  variance  es¬ 
timators,  the  denominator  estimating  a  variance  that  is  uninfluenced 
by  the  "amount**  of  multidimensionality  present  and  the  numerator 
estimating  a  variance  that  is  inflated  by  the  amount  of  mul ^dimen¬ 
sionality  present.  The  variance  estimators  are  each  based  upon  the 
first  subtest,  l.e.,  on  Items  1,  2,  ...,  M. 

5.  How,  fix  (k,n).  That  la,  look  at  the  persons  in  cell  k  of  the  I 
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nth  partition  ...,  A^V),  denoting  the  miaber  of 

partition  cello. 

6.  The  denominator  can  now  be  eonatructed.  Consider  item  a  (of  the 
flrat  aubtest,  hence,  1  <  a  <  M.  Let 

J(k) 

”  D.j/Jak>  *  112] 

J-l  J  " 

where  0^  Indicates  that  correctness  of  the  response  of  the  Jth  per¬ 
son  of  cell  k  to  item  m.  Let 


J  P(k)(l  -  P^/M2  , 

t  D  D 

m=l 


[13] 


the  denominator  estimator  of  variance.  Note  that  persons  have  been 

summed  over  first,  forming  p(^)  and  then  items,  forming  o|.  . 

(k)  ®  - 

7.  For  the  numerator,  let  gj  .be  the  proportion  correct  for  person 
on  the  first  subtest,  i.e. , 


(k) 

M 

8J 

*  E 

Let 

j(k> 

-(k) 

n 

8 

*  E 

J-l 

and 

j(k> 

«2 

n 

H  r 

°Rk 

*  L 

J-l 

(k) /.(k) 
8j  /Jn 


(g.(k)-?(k)‘ . 


[14] 


[15] 


[16] 


the  numerator  estimator  of  variance.  Note  that  items  have  been  sum¬ 
med  over  first,  forming  g^k^  and  then  persons,  forming  Og^. 

8.  For  the  estimator  let 


Fk* 


2 

Pk  * 


[17] 
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Thus,  for  each  cell  k,  a  statistic  Fk  is  obtained.  The  (Pk)  are 
Independent  random  variables. 


The  A sj 


>totlc  Distribution  of  {P^) 


In  order  to  use  the  {Fk)  to  make  Inferences  about  dimensionality,  their 

asymptotic  distribution  Is  needed.  To  this  end,  the  author  has  shown  that  for 
any  K  cells  Indexed  by  1,  2,  . ..,  X  there  exists  ck  >  0  such  that 


X 

E 


k-1  Ck 


[18] 


is  asymptotically  normal  with  mean  sero  and  variance  one  [notationally  N(0,1>) 
when  d  *  1  and,  moreover,  estimators  ck  of  ck  exist  such  that 


X 

£ 

k-1 


[19] 


is  asymptotically  N(0,1)  when  d  -  1.  Further,  It  has  been  shown  that  there  ex¬ 
ists  a  number  C  >  0  and  numbers  Ajj  k  _>  CM  such  that  in  probability  for 

k-1,  2,  ...,  X  when  d  >  1.  Hence,  there  exists  a  valid  large  sample  level  a 
procedure  for  testing  H  :d-  lvs.  A  :  d  >  1. 

It  also  follows  that  this  procedure  (even  in  the  extreme  case  of  K  -  1)  for  an 
appropriate  choice  of  M  has  asymptotic  power  one  for  any  fixed  alternative, 
l.e.,  any  distribution  of  6  for  which  d  >  1.  The  procedure  is  to  reject  H  if 

K  V1  /  r 

Z  /V*  >  Z«  *  1201 

k-1  / 

where  Z  Is  the  100  (1  -  a)  percentile  of  a  standard  normal  distribution. 


Discussion 

There  remain  several  important  theoretical  and  practical  questions  that 
should  be  investigated.  First,  there  are  clearly  several  plausible  ways  of  com¬ 
bining  the  F^'s  into  a  single  test  statistic  and  of  obtaining  the  asymptotic 

distribution  of  this  test  statistic.  Three  such  possibilities  are 


as  was  shown  above; 


I 
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where  I[A)  denotes  the  indicator  of  the  event  A;  and 
3*  A  chi-square  like  statistic  [1.(0.  -  based  upon  the  number 

of  k's  such  that 


The  author  plans  to  investigate  the  asymptotic  distributions  of  the  second  and 
third  of  these  statistics  as  veil. 

Second,  it  is  essential  to  carry  out  some  carefully  designed  monte  carlo 
studies  to  see  for  what  range  of  test  lengths  and  sample  6izes  of  examinees  the 
actual  distribution  of  the  F^'s  is  veil  approximated  by  the  asymptotic  distribu¬ 
tion  of  the  F^'s.  This  is  essential  because  asymptotic  distribution  theory  can¬ 
not  by  itself  guarantee  the  accuracy  of  the  approximation  that  it  suggests. 

Third,  the  meaningful  and  practical  question  is  not  vhether  d  ■  1  but, 
rather,  whether  taking  d  ■  1  accounts  for  most  of  the  explainable  regularity  in 
the  data.  Thus,  vhat  is  called  for  16  a  reformulation  of  the  hypothesis  that 
d  *  1  and  possibly  an  estimation  approach  in  order  to  estimate  hov  much  of  the 
explainable  regularity  is  accounted  for  by  taking  d  ■  1.  This  Important  practi¬ 
cal  concern  needs  to  be  dealt  vith  by  some  combination  of  a  theoretical  analysis 
and  a  monte  carlo  study. 

Fourth,  some  combination  of  a  theoretical  analysis  and  a  monte  carlo  study 
is  also  needed  so  that  some  quantitative  information  is  available  about  the 
pover  of  the  tests  constructed  from  the  F^'s. 

Fifth,  the  "regularity"  conditions  that  were  needed  on  the  rate  of  growth 

of  the  (J^)}  (numbers  of  persons  per  cell)  as  n  ®  in  order  to  establish  the 
n 

asymptotic  normality  of  the  F^'s— and,  hence,  the  asymptotic  distribution  of  the 
statistics  described  above— can  undoubtedly  be  Improved  upon.  This  would  fur¬ 
ther  strengthen  the  case  for  using  the  F^'s  in  actual  testing  situations.  More¬ 
over,  it  is  quite  possible  that  the  methods  of  proof  used  or  the  results  ob¬ 
tained  when  abstracted  from  the  present  situation  involving  the  F^'s  may  add  to 
the  general  body  of  knowledge  in  mathematical  statistics. 

Sixth,  the  procedures  that  are  obtained  from  carrying  out  the  above  should 
be  pilot  tested  on  actual  tests  and  populations. 

Seventh,  a  thorough  comparison  between  these  procedures  based  on  the  F^'s 

and  on  any  other  approaches  (such  as  factor  analytic)  in  the  literature  must  be 
made. 


L 
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Finally,  procedure*  should  also  be  developed  for  testing  H  :  d  •  k  vs.  A  : 
d  >  k  for  fixed  k  £  2.  Although  the  derivation  of  the  distribution  of  the  F^'s 

under  the  assumption  d  ■  1  was  surprisingly  delicate,  it  seems  clear  that  an 
analogous  procedure  for  this  hypothesis  testing  situation  can  be  found  and  its 
properties  studied. 

The  author  plans  to  investigate  these  questions  with  the  goal  of  producing 
a  theoretically  sound  and  practically  important  statistical  approach  to  the 
problem  of  making  Inferences  about  the  underlying  dimensionality. 
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Navy  Personnel  RID  Center 
San  Diego,  CA  92152 

1  Dr.  Norean  J.  Urr 
Chief  of  Naval  Technical  Training 
Naval  Air  Station  Neophis  (75) 
Millington,  TN  36054 

1  Dr.  Leonard  Kroeker 
Navy  Personnel  RID  Center 
San  liego,  CA  92152 


Navy 

1  Dr.  Nilliai  L.  Naloy  (02) 

Chief  of  Naval  Education  and  Training 
Naval  Air  Station 
Pensacola,  FL  32506 

1  Dr.  Kneale  Nar shall 
Chairean,  Operations  Research  Dept. 
Naval  Post  Graduate  School 
Ronterey,  CA  93940 

1  Dr.  Jaees  HcBride 
Navy  Personnel  RID  Center 
San  Diego,  CA  92152 

1  Cdr  Ralph  NcCueber 
Director,  Research  I  Analysis  Division 
Navy  Recruiting  Coeeand 
4015  Nil  son  Boulevard 
Arlington,  VA  22203 

1  Dr.  George  Noeller 
Director,  Behavioral  Sciences  Dept. 
Naval  Submarine  Nedical  Research  Lab 
Naval  Subeanne  Base 
Groton,  CT  06349 

1  Dr  Nilliai  Nontague 
NPRDC  Code  13 
San  Diego,  CA  92152 

1  Library,  Code  P201L 
Navy  Personnel  RID  Center 
San  Diego,  CA  92152 

1  Technical  Director 
Navy  Personnel  RID  Center 
San  Diego,  CA  92152 

6  Coaianding  Officer 
Naval  Research  Laboratory 
Code  2627 

Nashington,  DC  20390 

6  Personnel  I  Training  Research  Group 
Code  442PT 

Office  of  Naval  Research 
Arlington,  VA  22217 

1  LT  Frank  C.  Petho,  NSC,  DSN  (Ph.D) 

CNET  (N-432) 

NAS 

Pensacola,  FL  32500 
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Navy 

1  Dr.  Dernard  Rial  and  (01C) 

Navy  Personnel  RU  Cntar 
San  Ditto,  CD  92132 

1  Dr.  Carl  Rota 
CNET-PDCD 
lull dinq  90 

Brtat  Lakes  NTC,  IL  10088 

1  Nr.  Ore*  Sands 
NPRDC  Code  62 
San  Diego,  CA  92152 

1  Dr.  Robert  6.  Seith 
Office  of  Chief  of  Naval  Operations 
0P-9B7H 

Hashington,  DC  20330 

1  Dr.  Richard  Sno* 

Liaison  Scientist 
Office  of  Naval  Research 
Drench  Office,  London 
Box  39 

FPO  Ne*  fork,  NY  09310 

1  Dr.  Richard  Sorensen 
Navy  Personnel  RU)  Center 
San  Diego,  CA  92152 

1  Dr.  Frederick  Steinheiser 
CM)  -  0PU5 
Navy  Annex 
Arlington,  VA  20370 

1  Nr.  Drad  Syapson 
Navy  Personnel  RID  Center 
San  Diego,  CA  92132 

1  Dr.  Janes  Teeeddale 
Technical  Director 
Navy  Personnel  RID  Center 
San  Diego,  CA  92152 

1  Dr.  Ed*ard  Hegaan 

Office  of  Naval  Research  (Code  411SIP) 
BOO  North  Duincy  Street 
Arlington,  VA  22217 

1  Dr.  Douglas  Netzel 
Code  12 

Navy  Personnel  RfcD  Center 
San  Diego,  CA  92132 


Navy 

1  DR.  MARTIN  F.  NISKOFF 
NAVY  PERSONNEL  Rh  D  CENTER 
SAN  DIE60,  CA  92132 

1  Nr  John  H.  Nolfe 
Navy  Personnel  RED  Center 
San  Diego,  CA  92152 
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Narine  Corps 

I  H.  Nilliaa  6reenup 
Education  Advisor  (E031) 

Education  Center,  RCBEC 
Quantico,  VA  22134 

1  Jerry  Lehnus 
CAT  Project  Office 
HO  Marine  Corps 
Nashington  ,  DC  20380 

1  Director,  Office  of  Hanpouer  Util izatio 
HO,  Narine  Corps  (HPU) 

BCD,  Bldg.  2009 
Duantico,  VA  22134 

1  Headquarters,  U.  S.  Narine  Corps 
Code  NP I -20 
Hashing ton,  DC  203B0 

1  Special  Assistant  for  Narine 
Corps  Hatters 
Code  100N 

Office  of  Naval  Research 
800  N.  Ouincy  St. 

Arlington,  VA  22217 

1  DR.  A.L.  SLAFKOSKY 
SCIENTIFIC  ADVISOR  (CODE  RD-l) 

HO,  U.S.  NARINE  CORPS 
NASHIN6TDN,  DC  20380 

1  Major  Frank  Yohannan,  USHC 
Headquarters,  Narine  Corps 
(Code  HPI-20) 

Nashington,  DC  20380 


Aroy 

1  Technical  Director 
U.  S.  Aroy  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eiser  'uer  Avenue 
Alexandria,  VA  22333 

1  Dr.  Kent  Eaton 
Aray  Research  Institute 
5001  Eisenhouer  Blvd. 

Alexandria  ,  VA  22333 

I  Dr.  Hyron  Fischl 

U.S.  Aroy  Research  Institute  for  the 
Social  and  Behavioral  Sciences 
5001  Eisenhouer  Avenue 
Alexandria,  VA  22333 

1  Dr.  Hilton  S.  Katz 
Training  Technical  Area 
U.S.  Aray  Research  Institute 
5001  Eisenhouer  Avenue 
Alexandria,  VA  22333 

1  Dr.  Clessen  Martin 
Aray  Research  Institute 
5 001  iisenhoter  Blvd. 

Alexandria,  VA  22333 

1  Dr.  Nilliaa  E.  Nordbrock 
FHC-ADCO  Box  25 
APO,  NY  09710 

1  Nr.  Robert  Ross 

U.S.  Aray  Research  Institute  for  the 
Social  and  Behavioral  Sciences 
5001  Eisenhouer  Avenue 
Alexandria,  VA  22333 

1  Dr.  Robert  Sasaor 
U.  S.  Aray  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eisenhouer  Avenue 
Alexandria,  VA  22333 

1  Dr.  Joyce  Shields 
Aray  Research  Institute  for  the 
Behavioral  and  Social  Sciences 
5001  Eisenhouer  Avenue 
Alexandria,  VA  22333 

1  Dr.  Hilda  King 
Aray  Research  Institute 
5001  Eisenhouer  Ave. 

Alexandria,  VA  22333 
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Ai  r  Fore* 

1  Technical  Documents  C inter 
Air  Force  Huean  Resources  Laboratory 
WPAFB,  OK  45433 

1  U.S.  Air  Force  Office  of  Scientific 
Research 

Life  Sciences  Directorate,  HL 
Bolling  Air  Force  lose 
Washington,  DC  20332 

1  Air  University  Library 
AUL/LS £  76/443 
Haxiell  AFB,  AL  36112 

1  Dr,  Ear]  A.  Alluisi 
HO,  AFKRL  (AFSC) 

Brooks  AFB,  T1  78235 

1  Hr.  Rayeond  E.  Christal 
AFHRL/HOE 

Brooks  AF8,  TX  78235 

1  Dr,  Alfred  R.  Fregly 
AFOSR/HL 

Bolling  AFB,  DC  20332 

1  Dr.  Patrick  Kyllonen 
AFHRL/HOE 

Brooks  AFB,  TX  7B235 

1  Dr.  Roger  Pennell 
Air  Force  Huoan  Resources  Laboratory 
Loory  AFB,  CO  10230 

1  Dr.  Hal  coll  Ree 
AFHRL/HP 

Brooks  AFB,  TX  78235 


Departeent  of  Defense 

12  Defense  TKhnical  Inforution  Center 
Celeron  Station,  Bldg  5 
Alexandria,  VA  22314 
Attn:  TC 

1  Military  Assistant  for  Training  and 
Personnel  Technology 
Office  of  the  Under  Secretary  of  Defens 
for  Research  I  Engineering 
Rooi  31129,  The  Pentagon 
Washington,  DC  20301 

1  Dr.  W.  Steve  Sc Ilian 
Office  of  the  Assistant  Secretary 
of  Defense  (HRA  &  L) 

2B269  The  Pentagon 
Washington,  DC  20301 

1  Major  Jack  Thorpe 
DARPA 

1400  Wilson  Blvd. 

Arlington,  VA  22209 

1  Dr.  Robert  A.  Wisher 
OUSDRE  (ELS) 

The  Pentagon,  Root  3D129 
Washington,  DC  20301 
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Civilian  Agencies 

1  Or.  Vern  V.  Urry 
Personnel  010  Center 
Office  of  Personnel  Ranageoent 
1900  E  Street  IN 
Washington,  5C  20415 

1  Hr.  Thoaas  A.  Mara 
U.  S.  Coast  Guard  Institute 
P.  0.  Substation  18 
Oklahoma  City,  OK  73169 

1  Dr.  Frank  Dithroa 
U.  S.  Office  of  Education 
400  Haryland  Ave.  SW 
Washington,  DC  20202 

1  Dr.  Joseph  L.  Young,  Director 
Heeory  I  Cognitive  Processes 
National  Science  Foundation 
Washington,  DC  20550 


Private  Sector 

1  Dr.  Jam  Algina 
University  of  Florida 
Gainesville,  FL  326 

1  Dr.  Erling  1.  Andersen 
Department  of  Statistics 
Studiestraede  6 
1455  Copenhagen 
DENHAM 

1  1  Psychological  Research  Unit 
NBH-3-44  Attn 
Northbourne  House 
Turner  ACT  2601 
AUSTRALIA 

1  Dr.  Alan  Baddeley 
Hedical  Research  Council 
Applied  Psychology  Unit 
15  Chaucer  Road 
Cambridge  CB2  2£F 
ENGLAND 

1  Dr.  Isaac  Be jar 
Educational  Testing  Service 
Princeton,  NJ  0B450 

1  Dr.  Henucha  Birenbaua 
School  of  Education 
Tel  Aviv  University 
Tel  Aviv,  Raaat  Aviv  6997B 
Israel 

1  Dr.  R.  Darrell  Bock 
Departrent  of  Education 
University  of  Chicago 
Chicago,  IL  60637 

1  Dr.  Robert  Brennan 
American  College  Testing  Profits 
P.  0.  Box  168 
Iowa  City,  IA  52243 

1  Dr.  61enn  Bryan 
6208  Pee  Road 
Bethesda,  HD  20817 

1  Bundeinisteriue  der  Verteidigung 
-Refer at  P  II  4- 
Psychological  Service 
Postfach  132B 
D-5J00  Bonn  1 
F.  R.  of  6troany 


Ulinois/Levine 


l-Nir-M 


Page  6 


Private  Sector 


Private  Sector 


1  Dr.  Ernest  R.  Cadotte 
307  Stokely 

University  of  Tennessee 
Knoxville,  TN  37916 

1  Sr.  John  I.  Carroll 
409  Elliott  Rd. 

Chapel  Hill,  NC  27314 

1  Dr.  Norsan  Cliff 
Dept,  of  Psychology 
llniv.  of  So.  California 
University  Park 
Los  Rngeles,  CR  90007 

1  Dr.  Allan  fl.  Collins 
Dolt  Deranek  6  Neuean,  Inc. 
50  Hoolton  Street 
Caohridge,  HA  02136 


1  Dr.  Susan  Eohertson 
PSYCHOLOGY  DEPARTHENT 
UNIVERSITY  OF  KANSAS 
Laurence,  KS  66043 

1  ERIC  Facility-Acquisitions 
4833  Rugby  Avenue 
Bethesda,  HO  20014 

1  Dr.  Denjaain  A.  Fairbank,  Jr. 
HcFann-6ray  (  Associates,  Inc. 
3823  Callaghan 
Suite  223 

San  Antonio,  TI  78228 

1  Dr.  Leonard  Feldt 
Lindquist  Center  for  Neasureent 
University  of  Ioua 
Iona  City,  IA  32242 


1  Dr.  Lynn  A.  Cooper 
LRDC 

University  of  Pittsburgh 
3939  O’Hara  Street 
Pittsburgh,  PA  15213 

1  Dr.  Hans  Croebag 
Education  Research  Center 
University  of  Leyden 
Doerhaavelaan  2 
2334  EN  Leyden 
The  NETHERLANDS 

1  CTD/HcBrau-Hill  Library 
2300  Garden  Road 
Honterey,  CA  95940 

1  Dr.  Dattpradad  Divgi 
Syracuse  University 
Departaent  of  Psychology 
Syracuse,  HE  33210 

1  Dr.  Hei-Ki  Dong 
Dali  Foundation 
Rooa  314,  building  B 
800  Roosevelt  Road 
Glen  Ellyn,  IL  60137 

1  Dr.  Fritt  Drasgou 
Departaent  of  Psychology 
University  of  Illinois 
603  E.  Daniel  St. 
ClMpaign,  IL  61820 


1  Dr.  Richard  L.  Ferguson 
The  Aaerican  College  Testing  Prograa 

x  P.0.  Box  166 
Ioua  City,  IA  32240 

1  Dniv.  Prof.  Dr.  Gerhard  Fischer 
Liebiggasse  5/3 
A  1010  Vienna 
AUSTRIA 

1  Professor  Donald  Fitzgerald 
University  of  Neu  England 
Areidale,  Neu  South  Hales  2331 
AUSTRALIA 

1  Dr.  Dexter  Fletcher 
University  of  Oregon 
Departeent  of  Coaputer  Science 
Eugene,  OR  97403 

1  Dr.  John  R.  Frederiksen 
Dolt  Deranek  6  Neuaan 
SO  Houlton  Street 
Caobridge,  HA  02138 

1  Dr.  Janice  Gifford 
University  of  Hassachusetts 
School  of  Education 
Aaherst,  HA  01002 
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Private  Sector 

1  Or.  Robert  Blaser 

Learning  Research  I  Development  Center 
University  of  Pittsburgh 
3939  O’Hara  Street 
PITTSBUR6H,  PA  15260 

1  Or.  Bert  Breen 
Johns  Hopkins  University 
Department  of  Psychology 
Charles  I  34th  Street 
Baltimore,  HD  21218 

1  08.  JANES  6.  BREENO 
LRDC 

UNIVERSITY  Of  PITTSBURBH 
3939  O’HARA  STREET 
PITTSBURGH,  PA  15213 

1  Dr.  Ron  Hambieton 
School  of  Education 
University  of  Massachusetts 
Amherst,  HA  01002 

1  Dr.  Delvyn  Harnisch 
University  of  Illinois 
242b  Education 
Urbana,  IL  61801 

1  Dr.  Paul  Horst 
677  6  Street,  1184 
Chula  Vista,  CA  90010 

1  Dr.  Lloyd  Humphreys 
Department  of  Psychology 
University  of  Illinois 
603  East  Daniel  Street 
Champaign,  IL  61820 

1  Dr.  Steven  Hunk a 
Department  of  Education 
University  of  Alberta 
Edmonton,  Alberta 
CANADA 

1  Dr.  Earl  Hunt 
Dept,  of  Psychology 
University  of  Hashington 
Seattle,  HA  98105 

1  Dr.  Jack  Hunter 
2122  Coolidge  St. 

Lansing,  HI  48906 


Private  Sector 

1  Dr.  Huynh  Huynh 
College  of  Education 
University  of  South  Carolina 
Columbia,  SC  29206 

1  Dr.  Douglas  H.  Jones 
Advanced  Statistical  Technologies 
Corporation 
10  Trafalgar  Court 
Lawrenceville,  NJ  08148 

1  Professor  John  A.  Keats 
Department  of  Psychology 
The  University  of  Newcastle 
H.S.H.  2308 
AUSTRALIA 

1  Dr.  Scott  Kelso 
Haskins  Laboratories,  Inc 
270  Croun  Street 
Ne«  Haven,  CT  06510 

1  CDfi  Robert  S.  Kennedy 
Canyon  Research  Group 
1040  Hcodcock  Road 
Suite  227 
Orlando,  FL  32803 

1  Dr.  Hilliam  Koch 
University  of  Texas-Austin 
Heasurement  and  Evaluation  Center 
Austin,  TX  ?B703 

1  Dr.  Stephen  Kosslyn 
1236  Hilliam  James  Hall 
33  Kirkland  St. 

Cambridge,  HA  02138 

!  Dr.  Alan  Lesgold 
Learning  RID  Center 
University  of  Pittsburgh 
3939  O’Hara  Street 
Pittsburgh,  PA  15260 

1  Dr.  Hichael  Levine 
Bmpartrent  of  Educational  Psychology 
210  Education  Bldg. 

University  of  Illinois 
Champaign,  IL  61901 
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Privitf  Sector 

1  Dr.  Charles  Lems 
Faculteit  Soculc  Netenschappen 
Rijksuniversitmit  Broningtn 
(hide  loteringestraat  23 
971 26C  Broningen 
Netherlands 

1  Dr.  Robert  Linn 
College  of  Education 
University  of  Illinois 
Urbana,  IL  61801 

1  Hr.  Phillip  Livingston 
Systems  and  Applied  Sciences  Corporatio 
6811  Kenilworth  Avenue 
River dale,  RD  20840 

1  Dr.  Robert  Lockean 
Center  for  Naval  Analysis 
200  North  Beauregard  St. 

Alexandria,  VA  22311 

1  Dr.  Frederic  H.  Lord 
Educational  Testing  Service 
Princeton,  NJ  08541 

1  Dr.  James  Luesden 
Department  of  Psychology 
University  of  Nestern  Australia 
Nedlands'N.A.  6009 
AUSTRALIA 

1  Dr.  Don  Lyon 
P.  0.  Box  44 
Hi g ley  ,  A2  85236 

1  Dr.  Bary  Narco 
Stop  31-E 

Educational  Testing  Service 
Princeton,  NJ  08451 

1  Dr.  Scott  Raxwell 
Department  of  Psychology 
University  of  Notre  Dame 
Notre  Dame,  IN  46556 

1  Dr,  Samuel  T.  Kayo 
Loyola  University  »f  Chicago 
820  North  Michigan  Avenue 
Chicago,  IL  60611 


Private  Sector 

1  Nr.  Robert  McKinley 
American  College  Testing  Programs 
P.0,  kx  166 
loma  City,  I A  52243 

1  Dr.  Barbara  Means 
Human  Resources  Research  Organization 
300  North  Washington 
Alexandria,  VA  22314 


Professor  Jason  Hillman 
Department  of  Education 
Stone  Hall 
Cornell  University 
Ithaca,  NY  14853 

1  Dr.  Allen  Runro 

Behavioral  Technology  Laboratories 
1845  Eleia  Ave.,  Fourth  Floor 
Redondc  Beach,  CA  90277 

1  Dr.  N.  Alan  Nitewander 
University  of  Oklahoma 
Department  of  Psychology 
Oklahoma  City,  OK  73069 

1  Dr.  Donald  A  Ncraan 
Cognitive  Science,  C-015 
Uni v.  of  California,  San  Diego 
La  Jolla,  CA  92093 

1  Dr.  Helvin  R.  Kovick 
356  Lindquist  Center  f or  Reasurment 
University  of  Iowa 
loma  City,  1A  52242 

1  Dr.  James  Olson 
NICAT,  Inc. 

1875  Scuth  State  Street 
Ores,  UT  84057 

1  Dr.  Jesse  Orlansky 
Institute  for  Defense  Analyses 
1801  N.  Beauregard  St. 

Alexandria,  VA  22311 

1  Nayne  R.  Patience 
American  Council  on  Education 
BED  Testing  Service,  Suite  20 
One  Dupont  Cirle,  NN 
Nashington,  DC  20036 
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Private  Sector 

1  Or.  Jins  A.  Paulson 
Portland  State  University 
P.0,  loi  751 
Portland,  OP  97207 

1  Dr.  James  N.  Pellegrino 
University  of  California, 

Santa  Barbara 
Dept,  of  Psychology 
Santa  larabara  ,  CA  93106 

1  Dr.  Hark  D.  Reckase 
ACT 

P.  0.  Box  168 
Iowa  City,  I A  52243 

1  Dr.  Thoaas  Reynolds 
University  of  Texas-Dallas 
Harketing  Department 
P.  0.  Box  688 
Richardson,  TI  75080 

1  Dr.  Andrew  H.  Rose 
Aeerican  Institutes  for  Research 
1055  Thoaas  Jefferson  St.  NN 
Washington,  DC  20007 

1  Dr.  Ernst  Z.  Rothkopf 
Bell  Laboratories 
Hurray  Hill,  NJ  07974 

1  Dr.  Laurence  Rudner 
403  El  a  Avenue 
Takoma  Park,  HD  20012 

1  Dr.  J.  Ryan 
Department  of  Education 
University  of  South  Carolina 
Columbia,  SC  29208 

1  Frank  L.  Schmidt 
Department  of  Psychology 
Bldg.  66 

6eorge  Washington  University 
Washington,  DC  20052 

1  Dr.  Walter  Schneider 
Psychology  Department 
603  E.  Ban i el 
Champaign,  IL  61B20 


Private  Sector 

1  Lou ell  Schoer 
Psychological  A  Quantitative 
Foundations 
College  of  Education 
University  of  loea 
Iowa  City,  1A  52242 

1  Dr.  Kazuo  Shigemasu 
7-9-24  Kugenuaa-kaigan 
Fujusaua  251 
JAPAN 

1  Dr.  Edwin  Shirkey 
Department  of  Psychology 
University  of  Central  Florida 
Orlando,  FL  32816 

i  Dr.  William  Sims 
Center  for  Naval  Analysis 
200  North  Beauregard  Street 
Alexandria,  VA  22311 

1  Dr.  Robert  Sternberg 
Dept,  of  Psychology 
Yale  University 
Box  11A,  Yale  Station 
New  Haven,  CT  06520 

1  Hartha  Stocking 
Educational  Testing  Service 
Princeton,  NJ  08541 

1  Dr.  Peter  Stoloff 
Center  for  Naval  Analysis 
200  North  Beauregard  5treet 
Alexandria,  VA  22311 

1  David  E.  Stone,  Ph.D. 

Hazeltine  Corporation 
7680  Old  Springhouse  Road 
HcLean,  VA  22102 

1  Dr.  William  Stout 
University  of  Illinois 
Departient  of  Hathematics 
Urbana,  IL  61801 

1  BR.  PATRICK  SUPPES 
INSTITUTE  FOR  HATHEHATICAL  STUDIES  IN 
THE  SOCIAL  SCIENCES 
STANFORD  UNIVERSITY 
STANFORD,  CA  94305 
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Private  factor  Privitt  Smctor 


1  Or.  Han  bar  an  Smaoinathan 
Laboratory  of  Psychometric  and 
Evaluation  Research 
School  of  Education 
University  of  Massachusetts 
Amherst,  If  A  01003 

1  Dr.  Kikun  Tat&uoka 
Computer  Based  Education  Research  Lab 
252  Engineering  Research  Laboratory 
Urbana,  1L  61801 

1  Dr.  Maurice  Tatsuoka 
220  Education  Bldg 
1310  S.  Sixth  St. 

Champaign,  IL  61820 

1  Dr.  David  Thissen 
Department  of  Psychology 
University  of  Kansas 
Laurence,  KS  66044 

1  Dr.  Douglas  Tonne 
Univ.  of  So.  California 
Behavioral  Technology  Labs 
18^5  S,  Elena  Ave. 

Redondo  Beach,  CA  90277 

1  Dr.  Robert  Tsutakaua 
Department  of  Statistics 
University  of  Missouri 
Columbia,  HO  65201 

1  Dr.  V.  R.  R.  Uppuluri 
Union  Carbide  Corporation 
Nuclear  Bivision 
P.  0.  Box  Y 
Oak  Ridge,  TN  37830 

1  Dr.  David  Vale 
Assessment  Systems  Corporation 
2233  University  Avenue 
Suite  310 

St.  Paul,  MN  55114 

1  Dr.  K«rt  Van  Lehn 
lerox  PARC 

3333  Coyote  Hill  Road 
Palo  Alto,  CA  94304 


1  Dr.  Michael  T.  Nailer 
Departrent  of  Educational  Psychology 
University  of  Nisconsin—Miluaukee 
Hiluaukee,  HI  33201 

1  Dr.  Brian  Haters 
HuaRRO 

300  North  Nashington 
Alexandria,  VA  22314 

1  Dr.  David  J.  Nms 
H660  Elliott  Hall 
University  of  Minnesota 
75  E.  River  Road 
Minneapolis,  MM  55453 

1  Dr.  Donald  l.  Neitzman 
Mitre  Corporation 
1820  Del  ley  Madison  Blvd 
McLean,  VA  22102 
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